Efficient point-in-polygon indexing technique to facilitate displaying geographic data

ABSTRACT

A system that displays geographic data is disclosed. The system obtains polygons that define a set of geographic regions. Then, the system projects rays from endpoints of the line segments that define the polygons onto a reference line to form intersection points. For each interval between pairs of consecutive intersection points on the reference line, the system keeps track of open line segments that project onto the interval. For each data point in a set of data points, the system identifies a relevant interval on the reference line that the data point projects onto, and performs a crossing number operation to identify polygons that the data point falls into, and the system increments a count for each polygon that the data point falls into. Finally, the system displays the set of geographic regions in a manner that indicates a number of data points that fall into each geographic region.

RELATED APPLICATION

This application is related to pending U.S. patent application Ser. No.TBA filed on the same day as the instant application, entitled“Efficient Point-In-Polygon Indexing Technique for Processing QueriesOver Geographic Data Sets” by inventor Geoffrey R. Hendry (AttorneyDocket No. SPLK14-1028). This application is also related to pendingU.S. patent application Ser. No. TBA filed on the same day as theinstant application, entitled “Efficient Point-In-Polygon IndexingTechnique for Facilitating Geofencing Operations” by inventor GeoffreyR. Hendry (Attorney Docket No. SPLK14-1034).

BACKGROUND

1. Field of the Invention

The disclosed embodiments generally relate to techniques for queryingand viewing geographic data sets. More specifically, the disclosedembodiments relate to a point-in-polygon indexing technique tofacilitate viewing statistical data associated with geographic regions.

2. Related Art

The point-in-polygon (PIP) problem has important applications in avariety of disciplines, including geographic information systems (GISs),computer games, computer vision applications and computer-aided design(CAD). Solutions to the PIP problem have been motivated by therequirements of choropleth maps for geographic information systems(GISs). A choropleth map is a type of “heat map” in which polygons areshaded based on the quantity or density of certain subjects of interestwithin the polygon. For example, one can count the number of Starbucks™stores within each state (the state boundaries being the set ofpolygons), and then visually present the information by filling in eachpolygon with a color or hue that reflects the count of Starbucks™ storesin the given polygon. In this example, the digitized boundaries of theUS states may contain hundreds of thousands of points for each state.Hence, application of a brute force crossing number (CN) technique tosolve the PIP problem involves iterating over all of the many verticesin each state boundary. While some performance optimizations exist forthe special case of non-overlapping polygons, in general suchoptimizations are impossible or impractical. Furthermore, the CNtechnique needs to be run on each data point. Consequently, processingeven a small number of data points against a large GIS boundary datasetis computationally expensive.

Hence, what is needed is a system that facilitates efficiently solvingthe PIP problem without the limitations of existing techniques.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 presents a block diagram of an event-processing system inaccordance with the disclosed embodiments.

FIG. 2 presents a flowchart illustrating how indexers process, index,and store data received from forwarders in accordance with the disclosedembodiments.

FIG. 3 presents a flowchart illustrating how a search head and indexersperform a search query in accordance with the disclosed embodiments.

FIG. 4 presents a block diagram of a system for processing searchrequests that uses extraction rules for field values in accordance withthe disclosed embodiments.

FIG. 5 illustrates an exemplary search query received from a client andexecuted by search peers in accordance with the disclosed embodiments.

FIG. 6A illustrates a search screen in accordance with the disclosedembodiments.

FIG. 6B illustrates a data summary dialog that enables a user to selectvarious data sources in accordance with the disclosed embodiments.

FIG. 7 illustrates a system for processing and displaying geographicdata in accordance with the disclosed embodiments.

FIG. 8 illustrates a ray being cast across a polygon with crossingnumber (CN) counts in accordance with the disclosed embodiments.

FIG. 9 illustrates polygon segments projected onto a y-axis inaccordance with the disclosed embodiments.

FIG. 10 illustrates horizontal rays cast in each Y-range in accordancewith the disclosed embodiments.

FIG. 11 illustrates segment projections with multiple polygons inaccordance with the disclosed embodiments.

FIG. 12 illustrates a grouping of intersection sets by polygon identityin accordance with the disclosed embodiments.

FIG. 13 presents a flow chart illustrating the process of displayinggeographic data in accordance with the disclosed embodiments.

FIG. 14 presents a flow chart illustrating the process of performing ageofencing operation in accordance with the disclosed embodiments.

FIG. 15 presents a flow chart illustrating the processing of a queryinvolving geographic data in accordance with the disclosed embodiments.

Table 1 illustrates sorted point structures in accordance with thedisclosed embodiments.

Table 2 illustrates sorted point structures with corresponding lists ofopen segments in accordance with the disclosed embodiments.

Table 3 illustrates how non-final rows can be struck out in accordancewith the disclosed embodiments.

Table 4 illustrates Y ranges and corresponding open segments inaccordance with the disclosed embodiments.

Table 5 illustrates how non-final rows can be struck out in accordancewith the disclosed embodiments.

Table 6 illustrates how the data structure can be modified for modulo 3storage in accordance with the disclosed embodiments.

DETAILED DESCRIPTION

The disclosed embodiments relate to a system that facilitates queryingand displaying geographic data. This system is described in more detailbelow, but we first describe the structure of an event-based frameworkin which this system operates.

1.1 System Overview

Modern data centers often comprise thousands of host computer systemsthat operate collectively to service requests from even larger numbersof remote clients. During operation, these data centers generatesignificant volumes of performance data and diagnostic information thatcan be analyzed to quickly diagnose performance problems. In order toreduce the size of this performance data, the data is typicallypre-processed prior to being stored based on anticipated data-analysisneeds. For example, pre-specified data items can be extracted from theperformance data and stored in a database to facilitate efficientretrieval and analysis at search time. However, the rest of theperformance data is not saved and is essentially discarded duringpre-processing. As storage capacity becomes progressively cheaper andmore plentiful, there are fewer incentives to discard this performancedata and many reasons to keep it.

This plentiful storage capacity is presently making it feasible to storemassive quantities of minimally processed performance data at “ingestiontime” for later retrieval and analysis at “search time.” Note thatperforming the analysis operations at search time provides greaterflexibility because it enables an analyst to search all of theperformance data, instead of searching pre-specified data items thatwere stored at ingestion time. This enables the analyst to investigatedifferent aspects of the performance data instead of being confined tothe pre-specified set of data items that was selected at ingestion time.

However, analyzing massive quantities of heterogeneous performance dataat search time can be a challenging task. A data center may generateheterogeneous performance data from thousands of different components,which can collectively generate tremendous volumes of performance datathat can be time-consuming to analyze. For example, this performancedata can include data from system logs, network packet data, sensordata, and data generated by various applications. Also, the unstructurednature of much of this performance data can pose additional challengesbecause of the difficulty of applying semantic meaning to unstructureddata, and the difficulty of indexing and querying unstructured datausing traditional database systems.

These challenges can be addressed by using an event-based system, suchas the SPLUNK® ENTERPRISE system produced by Splunk Inc. of SanFrancisco, Calif., to store and process performance data. The SPLUNK®ENTERPRISE system is the leading platform for providing real-timeoperational intelligence that enables organizations to collect, index,and harness machine-generated data from various websites, applications,servers, networks, and mobile devices that power their businesses. TheSPLUNK® ENTERPRISE system is particularly useful for analyzingunstructured performance data, which is commonly found in system logfiles. Although many of the techniques described herein are explainedwith reference to the SPLUNK® ENTERPRISE system, the techniques are alsoapplicable to other types of data server systems.

In the SPLUNK® ENTERPRISE system, performance data is stored as“events,” wherein each event comprises a collection of performance dataand/or diagnostic information that is generated by a computer system andis correlated with a specific point in time. Events can be derived from“time series data,” wherein time series data comprises a sequence ofdata points (e.g., performance measurements from a computer system) thatare associated with successive points in time and are typically spacedat uniform time intervals. Events can also be derived from “structured”or “unstructured” data. Structured data has a predefined format, whereinspecific data items with specific data formats reside at predefinedlocations in the data. For example, structured data can include dataitems stored in fields in a database table. In contrast, unstructureddata does not have a predefined format. This means that unstructureddata can comprise various data items having different data types thatcan reside at different locations. For example, when the data source isan operating system log, an event can include one or more lines from theoperating system log containing raw data that includes different typesof performance and diagnostic information associated with a specificpoint in time. Examples of data sources from which an event may bederived include, but are not limited to: web servers; applicationservers; databases; firewalls; routers; operating systems; and softwareapplications that execute on computer systems, mobile devices, andsensors. The data generated by such data sources can be produced invarious forms including, for example and without limitation, server logfiles, activity log files, configuration files, messages, network packetdata, performance measurements and sensor measurements. An eventtypically includes a timestamp that may be derived from the raw data inthe event, or may be determined through interpolation between temporallyproximate events having known timestamps.

The SPLUNK® ENTERPRISE system also facilitates using a flexible schemato specify how to extract information from the event data, wherein theflexible schema may be developed and redefined as needed. Note that aflexible schema may be applied to event data “on the fly,” when it isneeded (e.g., at search time), rather than at ingestion time of the dataas in traditional database systems. Because the schema is not applied toevent data until it is needed (e.g., at search time), it is referred toas a “late-binding schema.”

During operation, the SPLUNK® ENTERPRISE system starts with raw data,which can include unstructured data, machine data, performancemeasurements or other time-series data, such as data obtained fromweblogs, syslogs, or sensor readings. It divides this raw data into“portions,” and optionally transforms the data to produce timestampedevents. The system stores the timestamped events in a data store, andenables a user to run queries against the data store to retrieve eventsthat meet specified criteria, such as containing certain keywords orhaving specific values in defined fields. Note that the term “field”refers to a location in the event data containing a value for a specificdata item.

As noted above, the SPLUNK® ENTERPRISE system facilitates using alate-binding schema while performing queries on events. A late-bindingschema specifies “extraction rules” that are applied to data in theevents to extract values for specific fields. More specifically, theextraction rules for a field can include one or more instructions thatspecify how to extract a value for the field from the event data. Anextraction rule can generally include any type of instruction forextracting values from data in events. In some cases, an extraction rulecomprises a regular expression, in which case the rule is referred to asa “regex rule.”

In contrast to a conventional schema for a database system, alate-binding schema is not defined at data ingestion time. Instead, thelate-binding schema can be developed on an ongoing basis until the timea query is actually executed. This means that extraction rules for thefields in a query may be provided in the query itself, or may be locatedduring execution of the query. Hence, as an analyst learns more aboutthe data in the events, the analyst can continue to refine thelate-binding schema by adding new fields, deleting fields, or changingthe field extraction rules until the next time the schema is used by aquery. Because the SPLUNK® ENTERPRISE system maintains the underlyingraw data and provides a late-binding schema for searching the raw data,it enables an analyst to investigate questions that arise as the analystlearns more about the events.

In the SPLUNK® ENTERPRISE system, a field extractor may be configured toautomatically generate extraction rules for certain fields in the eventswhen the events are being created, indexed, or stored, or possibly at alater time. Alternatively, a user may manually define extraction rulesfor fields using a variety of techniques.

Also, a number of “default fields” that specify metadata about theevents rather than data in the events themselves can be createdautomatically. For example, such default fields can specify: a timestampfor the event data; a host from which the event data originated; asource of the event data; and a source type for the event data. Thesedefault fields may be determined automatically when the events arecreated, indexed or stored.

In some embodiments, a common field name may be used to reference two ormore fields containing equivalent data items, even though the fields maybe associated with different types of events that possibly havedifferent data formats and different extraction rules. By enabling acommon field name to be used to identify equivalent fields fromdifferent types of events generated by different data sources, thesystem facilitates use of a “common information model” (CIM) across thedifferent data sources.

1.2 Data Server System

FIG. 1 presents a block diagram of an exemplary event-processing system100, similar to the SPLUNK® ENTERPRISE system. System 100 includes oneor more forwarders 101 that collect data obtained from a variety ofdifferent data sources 105, and one or more indexers 102 that store,process, and/or perform operations on this data, wherein each indexeroperates on data contained in a specific data store 103. Theseforwarders and indexers can comprise separate computer systems in a datacenter, or may alternatively comprise separate processes executing onvarious computer systems in a data center.

During operation, the forwarders 101 identify which indexers 102 willreceive the collected data and then forward the data to the identifiedindexers. Forwarders 101 can also perform operations to strip outextraneous data and detect timestamps in the data. The forwarders nextdetermine which indexers 102 will receive each data item and thenforward the data items to the determined indexers 102.

Note that distributing data across different indexers facilitatesparallel processing. This parallel processing can take place at dataingestion time, because multiple indexers can process the incoming datain parallel. The parallel processing can also take place at search time,because multiple indexers can search through the data in parallel.

System 100 and the processes described below with respect to FIGS. 1-5are further described in “Exploring Splunk Search Processing Language(SPL) Primer and Cookbook” by David Carasso, UM Research, 2012, and in“Optimizing Data Analysis With a Semi-Structured Time Series Database”by Ledion Bitincka, Archana Ganapathi, Stephen Sorkin, and Steve Zhang,SLAML, 2010, each of which is hereby incorporated herein by reference inits entirety for all purposes.

1.3 Data Ingestion

FIG. 2 presents a flow chart illustrating how an indexer processes,indexes, and stores data received from forwarders in accordance with thedisclosed embodiments. At block 201, the indexer receives the data fromthe forwarder. Next, at block 202, the indexer apportions the data intoevents. Note that the data can include lines of text that are separatedby carriage returns or line breaks and an event may include one or moreof these lines. During the apportioning process, the indexer can useheuristic rules to automatically determine the boundaries of the events,which for example coincide with line boundaries. These heuristic rulesmay be determined based on the source of the data, wherein the indexercan be explicitly informed about the source of the data or can infer thesource of the data by examining the data. These heuristic rules caninclude regular expression-based rules or delimiter-based rules fordetermining event boundaries, wherein the event boundaries may beindicated by predefined characters or character strings. Thesepredefined characters may include punctuation marks or other specialcharacters including, for example, carriage returns, tabs, spaces orline breaks. In some cases, a user can fine-tune or configure the rulesthat the indexers use to determine event boundaries in order to adaptthe rules to the user's specific requirements.

Next, the indexer determines a timestamp for each event at block 203. Asmentioned above, these timestamps can be determined by extracting thetime directly from data in the event, or by interpolating the time basedon timestamps from temporally proximate events. In some cases, atimestamp can be determined based on the time the data was received orgenerated. The indexer subsequently associates the determined timestampwith each event at block 204, for example by storing the timestamp asmetadata for each event.

Then, the system can apply transformations to data to be included inevents at block 205. For log data, such transformations can includeremoving a portion of an event (e.g., a portion used to define eventboundaries, extraneous text, characters, etc.) or removing redundantportions of an event. Note that a user can specify portions to beremoved using a regular expression or any other possible technique.

Next, a keyword index can optionally be generated to facilitate fastkeyword searching for events. To build a keyword index, the indexerfirst identifies a set of keywords in block 206. Then, at block 207 theindexer includes the identified keywords in an index, which associateseach stored keyword with references to events containing that keyword(or to locations within events where that keyword is located). When anindexer subsequently receives a keyword-based query, the indexer canaccess the keyword index to quickly identify events containing thekeyword.

In some embodiments, the keyword index may include entries forname-value pairs found in events, wherein a name-value pair can includea pair of keywords connected by a symbol, such as an equals sign orcolon. In this way, events containing these name-value pairs can bequickly located. In some embodiments, fields can automatically begenerated for some or all of the name-value pairs at the time ofindexing. For example, if the string “dest=10.0.1.2” is found in anevent, a field named “dest” may be created for the event, and assigned avalue of “10.0.1.2.”

Finally, the indexer stores the events in a data store at block 208,wherein a timestamp can be stored with each event to facilitatesearching for events based on a time range. In some cases, the storedevents are organized into a plurality of buckets, wherein each bucketstores events associated with a specific time range. This not onlyimproves time-based searches, but it also allows events with recenttimestamps that may have a higher likelihood of being accessed to bestored in faster memory to facilitate faster retrieval. For example, abucket containing the most recent events can be stored as flash memoryinstead of on hard disk.

Each indexer 102 is responsible for storing and searching a subset ofthe events contained in a corresponding data store 103. By distributingevents among the indexers and data stores, the indexers can analyzeevents for a query in parallel, for example using map-reduce techniques,wherein each indexer returns partial responses for a subset of events toa search head that combines the results to produce an answer for thequery. By storing events in buckets for specific time ranges, an indexermay further optimize searching by looking only in buckets for timeranges that are relevant to a query.

Moreover, events and buckets can also be replicated across differentindexers and data stores to facilitate high availability and disasterrecovery as is described in U.S. patent application Ser. No. 14/266,812filed on 30 Apr. 2014, and in U.S. patent application Ser. No.14/266,817 also filed on 30 Apr. 2014.

1.4 Query Processing

FIG. 3 presents a flow chart illustrating how a search head and indexersperform a search query in accordance with the disclosed embodiments. Atthe start of this process, a search head receives a search query from aclient at block 301. Next, at block 302, the search head analyzes thesearch query to determine what portions can be delegated to indexers andwhat portions need to be executed locally by the search head. At block303, the search head distributes the determined portions of the query tothe indexers. Note that commands that operate on single events can betrivially delegated to the indexers, while commands that involve eventsfrom multiple indexers are harder to delegate.

Then, at block 304, the indexers to which the query was distributedsearch their data stores for events that are responsive to the query. Todetermine which events are responsive to the query, the indexer searchesfor events that match the criteria specified in the query. This criteriacan include matching keywords or specific values for certain fields. Ina query that uses a late-binding schema, the searching operations inblock 304 may involve using the late-binding scheme to extract valuesfor specified fields from events at the time the query is processed.Next, the indexers can either send the relevant events back to thesearch head, or use the events to calculate a partial result, and sendthe partial result back to the search head.

Finally, at block 305, the search head combines the partial resultsand/or events received from the indexers to produce a final result forthe query. This final result can comprise different types of datadepending upon what the query is asking for. For example, the finalresults can include a listing of matching events returned by the query,or some type of visualization of data from the returned events. Inanother example, the final result can include one or more calculatedvalues derived from the matching events.

Moreover, the results generated by system 100 can be returned to aclient using different techniques. For example, one technique streamsresults back to a client in real-time as they are identified. Anothertechnique waits to report results to the client until a complete set ofresults is ready to return to the client. Yet another technique streamsinterim results back to the client in real-time until a complete set ofresults is ready, and then returns the complete set of results to theclient. In another technique, certain results are stored as “searchjobs,” and the client may subsequently retrieve the results byreferencing the search jobs.

The search head can also perform various operations to make the searchmore efficient. For example, before the search head starts executing aquery, the search head can determine a time range for the query and aset of common keywords that all matching events must include. Next, thesearch head can use these parameters to query the indexers to obtain asuperset of the eventual results. Then, during a filtering stage, thesearch head can perform field-extraction operations on the superset toproduce a reduced set of search results.

1.5 Field Extraction

FIG. 4 presents a block diagram illustrating how fields can be extractedduring query processing in accordance with the disclosed embodiments. Atthe start of this process, a search query 402 is received at a queryprocessor 404. Query processor 404 includes various mechanisms forprocessing a query, wherein these mechanisms can reside in a search head104 and/or an indexer 102. Note that the exemplary search query 402illustrated in FIG. 4 is expressed in the Search Processing Language(SPL), which is used in conjunction with the SPLUNK® ENTERPRISE system.SPL is a pipelined search language in which a set of inputs is operatedon by a first command in a command line, and then a subsequent commandfollowing the pipe symbol “I” operates on the results produced by thefirst command, and so on for additional commands. Search query 402 canalso be expressed in other query languages, such as the Structured QueryLanguage (“SQL”) or any suitable query language.

Upon receiving search query 402, query processor 404 sees that searchquery 402 includes two fields “IP” and “target.” Query processor 404also determines that the values for the “IP” and “target” fields havenot already been extracted from events in data store 414, andconsequently determines that query processor 404 needs to use extractionrules to extract values for the fields. Hence, query processor 404performs a lookup for the extraction rules in a rule base 406, whereinrule base 406 maps field names to corresponding extraction rules andobtains extraction rules 408-409, wherein extraction rule 408 specifieshow to extract a value for the “IP” field from an event, and extractionrule 409 specifies how to extract a value for the “target” field from anevent. As is illustrated in FIG. 4, extraction rules 408-409 cancomprise regular expressions that specify how to extract values for therelevant fields. Such regular-expression-based extraction rules are alsoreferred to as “regex rules.” In addition to specifying how to extractfield values, the extraction rules may also include instructions forderiving a field value by performing a function on a character string orvalue retrieved by the extraction rule. For example, a transformationrule may truncate a character string, or convert the character stringinto a different data format. In some cases, the query itself canspecify one or more extraction rules.

Next, query processor 404 sends extraction rules 408-409 to a fieldextractor 412, which applies extraction rules 408-409 to events 416-418in a data store 414. Note that data store 414 can include one or moredata stores, and extraction rules 408-409 can be applied to largenumbers of events in data store 414, and are not meant to be limited tothe three events 416-418 illustrated in FIG. 4. Moreover, the queryprocessor 404 can instruct field extractor 412 to apply the extractionrules to all the events in a data store 414, or to a subset of theevents that has been filtered based on some criteria.

Next, field extractor 412 applies extraction rule 408 for the firstcommand “Search IP=“10*” to events in data store 414 including events416-418. Extraction rule 408 is used to extract values for the IPaddress field from events in data store 414 by looking for a pattern ofone or more digits, followed by a period, followed again by one or moredigits, followed by another period, followed again by one or moredigits, followed by another period, and followed again by one or moredigits. Next, field extractor 412 returns field values 420 to queryprocessor 404, which uses the criterion IP=“10*” to look for IPaddresses that start with “10”. Note that events 416 and 417 match thiscriterion, but event 418 does not, so the result set for the firstcommand is events 416-417.

Query processor 404 then sends events 416-417 to the next command “statscount target.” To process this command, query processor 404 causes fieldextractor 412 to apply extraction rule 409 to events 416-417. Extractionrule 409 is used to extract values for the target field for events416-417 by skipping the first four commas in events 416-417, and thenextracting all of the following characters until a comma or period isreached. Next, field extractor 412 returns field values 421 to queryprocessor 404, which executes the command “stats count target” to countthe number of unique values contained in the target fields, which inthis example produces the value “2” that is returned as a final result422 for the query.

Note that query results can be returned to a client, a search head, orany other system component for further processing. In general, queryresults may include: a set of one or more events; a set of one or morevalues obtained from the events; a subset of the values; statisticscalculated based on the values; a report containing the values; or avisualization, such as a graph or chart, generated from the values.

1.6 Exemplary Search Screen

FIG. 6A illustrates an exemplary search screen 600 in accordance withthe disclosed embodiments. Search screen 600 includes a search bar 602that accepts user input in the form of a search string. It also includesa time range picker 612 that enables the user to specify a time rangefor the search. For “historical searches” the user can select a specifictime range, or alternatively a relative time range, such as “today,”“yesterday” or “last week.” For “real-time searches,” the user canselect the size of a preceding time window to search for real-timeevents. Search screen 600 also initially displays a “data summary”dialog as is illustrated in FIG. 6B that enables the user to selectdifferent sources for the event data, for example by selecting specifichosts and log files.

After the search is executed, the search screen 600 can display theresults through search results tabs 604, wherein search results tabs 604includes: an “events tab” that displays various information about eventsreturned by the search; a “statistics tab” that displays statisticsabout the search results; and a “visualization tab” that displaysvarious visualizations of the search results. The events tab illustratedin FIG. 6A displays a timeline graph 605 that graphically illustratesthe number of events that occurred in one-hour intervals over theselected time range. It also displays an events list 608 that enables auser to view the raw data in each of the returned events. Itadditionally displays a fields sidebar 606 that includes statisticsabout occurrences of specific fields in the returned events, including“selected fields” that are pre-selected by the user, and “interestingfields” that are automatically selected by the system based onpre-specified criteria.

1.7 Acceleration Techniques

The above-described system provides significant flexibility by enablinga user to analyze massive quantities of minimally processed performancedata “on the fly” at search time instead of storing pre-specifiedportions of the performance data in a database at ingestion time. Thisflexibility enables a user to see correlations in the performance dataand perform subsequent queries to examine interesting aspects of theperformance data that may not have been apparent at ingestion time.

However, performing extraction and analysis operations at search timecan involve a large amount of data and require a large number ofcomputational operations, which can cause considerable delays whileprocessing the queries. Fortunately, a number of acceleration techniqueshave been developed to speed up analysis operations performed at searchtime. These techniques include: (1) performing search operations inparallel by formulating a search as a map-reduce computation; (2) usinga keyword index; (3) using a high performance analytics store; and (4)accelerating the process of generating reports. These techniques aredescribed in more detail below.

1.7.1 Map-Reduce Technique

To facilitate faster query processing, a query can be structured as amap-reduce computation, wherein the “map” operations are delegated tothe indexers, while the corresponding “reduce” operations are performedlocally at the search head. For example, FIG. 5 illustrates how a searchquery 501 received from a client at search head 104 can split into twophases, including: (1) a “map phase” comprising subtasks 502 (e.g., dataretrieval or simple filtering) that may be performed in parallel and are“mapped” to indexers 102 for execution, and (2) a “reduce phase”comprising a merging operation 503 to be executed by the search headwhen the results are ultimately collected from the indexers.

During operation, upon receiving search query 501, search head 104modifies search query 501 by substituting “stats” with “prestats” toproduce search query 502, and then distributes search query 502 to oneor more distributed indexers, which are also referred to as “searchpeers.” Note that search queries may generally specify search criteriaor operations to be performed on events that meet the search criteria.Search queries may also specify field names, as well as search criteriafor the values in the fields or operations to be performed on the valuesin the fields. Moreover, the search head may distribute the full searchquery to the search peers as is illustrated in FIG. 3, or mayalternatively distribute a modified version (e.g., a more restrictedversion) of the search query to the search peers. In this example, theindexers are responsible for producing the results and sending them tothe search head. After the indexers return the results to the searchhead, the search head performs the merging operations 503 on theresults. Note that by executing the computation in this way, the systemeffectively distributes the computational operations while minimizingdata transfers.

1.7.2 Keyword Index

As described above with reference to the flow charts in FIGS. 2 and 3,event-processing system 100 can construct and maintain one or morekeyword indices to facilitate rapidly identifying events containingspecific keywords. This can greatly speed up the processing of queriesinvolving specific keywords. As mentioned above, to build a keywordindex, an indexer first identifies a set of keywords. Then, the indexerincludes the identified keywords in an index, which associates eachstored keyword with references to events containing that keyword, or tolocations within events where that keyword is located. When an indexersubsequently receives a keyword-based query, the indexer can access thekeyword index to quickly identify events containing the keyword.

1.7.3 High Performance Analytics Store

To speed up certain types of queries, some embodiments of system 100make use of a high performance analytics store, which is referred to asa “summarization table,” that contains entries for specific field-valuepairs. Each of these entries keeps track of instances of a specificvalue in a specific field in the event data and includes references toevents containing the specific value in the specific field. For example,an exemplary entry in a summarization table can keep track ofoccurrences of the value “94107” in a “ZIP code” field of a set ofevents, wherein the entry includes references to all of the events thatcontain the value “94107” in the ZIP code field. This enables the systemto quickly process queries that seek to determine how many events have aparticular value for a particular field, because the system can examinethe entry in the summarization table to count instances of the specificvalue in the field without having to go through the individual events ordo extractions at search time. Also, if the system needs to process allevents that have a specific field-value combination, the system can usethe references in the summarization table entry to directly access theevents to extract further information without having to search all ofthe events to find the specific field-value combination at search time.

In some embodiments, the system maintains a separate summarization tablefor each of the above-described time-specific buckets that stores eventsfor a specific time range, wherein a bucket-specific summarization tableincludes entries for specific field-value combinations that occur inevents in the specific bucket. Alternatively, the system can maintain aseparate summarization table for each indexer, wherein theindexer-specific summarization table only includes entries for theevents in a data store that is managed by the specific indexer.

The summarization table can be populated by running a “collection query”that scans a set of events to find instances of a specific field-valuecombination, or alternatively instances of all field-value combinationsfor a specific field. A collection query can be initiated by a user, orcan be scheduled to occur automatically at specific time intervals. Acollection query can also be automatically launched in response to aquery that asks for a specific field-value combination.

In some cases, the summarization tables may not cover all of the eventsthat are relevant to a query. In this case, the system can use thesummarization tables to obtain partial results for the events that arecovered by summarization tables, but may also have to search throughother events that are not covered by the summarization tables to produceadditional results. These additional results can then be combined withthe partial results to produce a final set of results for the query.This summarization table and associated techniques are described in moredetail in U.S. Pat. No. 8,682,925, issued on Mar. 25, 2014.

1.7.4 Accelerating Report Generation

In some embodiments, a data server system such as the SPLUNK® ENTERPRISEsystem can accelerate the process of periodically generating updatedreports based on query results. To accelerate this process, asummarization engine automatically examines the query to determinewhether generation of updated reports can be accelerated by creatingintermediate summaries. (This is possible if results from preceding timeperiods can be computed separately and combined to generate an updatedreport. In some cases, it is not possible to combine such incrementalresults, for example where a value in the report depends onrelationships between events from different time periods.) If reportscan be accelerated, the summarization engine periodically generates asummary covering data obtained during a latest non-overlapping timeperiod. For example, where the query seeks events meeting a specifiedcriteria, a summary for the time period includes only events within thetime period that meet the specified criteria. Similarly, if the queryseeks statistics calculated from the events, such as the number ofevents that match the specified criteria, then the summary for the timeperiod includes the number of events in the period that match thespecified criteria.

In parallel with the creation of the summaries, the summarization engineschedules the periodic updating of the report associated with the query.During each scheduled report update, the query engine determines whetherintermediate summaries have been generated covering portions of the timeperiod covered by the report update. If so, then the report is generatedbased on the information contained in the summaries. Also, if additionalevent data has been received and has not yet been summarized, and isrequired to generate the complete report, the query can be run on thisadditional event data. Then, the results returned by this query on theadditional event data, along with the partial results obtained from theintermediate summaries, can be combined to generate the updated report.This process is repeated each time the report is updated. Alternatively,if the system stores events in buckets covering specific time ranges,then the summaries can be generated on a bucket-by-bucket basis. Notethat producing intermediate summaries can save the work involved inre-running the query for previous time periods, so only the newer eventdata needs to be processed while generating an updated report. Thesereport acceleration techniques are described in more detail in U.S. Pat.No. 8,589,403, issued on Nov. 19, 2013, and U.S. Pat. No. 8,412,696,issued on Apr. 2, 2011.

System for Processing and Displaying Geographic Data

FIG. 7 illustrates a system for processing and displaying geographicdata in accordance with the disclosed embodiments. This system includesa data store 103 containing geographic data, such as populationstatistics associated with geographic regions. It also includes a queryprocessor 710 configured to execute queries involving geographic data.Results generated by query processor 710 can be formatted using displaymechanism 708 and then displayed to a user 702 through a display system704. For example, as illustrated in FIG. 7, the results can be displayedusing a choropleth map 706, wherein each geographic region is shaded,patterned or colored in proportion to a number of data points that fallinto the geographic region. The process of generating a choropleth map(or some other representation of geographic data) is described in moredetail below.

Fast Point-in-Polygon Indexing

The point-in-polygon (PIP) problem is typically solved using either thecrossing number (CN) technique or the winding number (WN) technique. Forexample, the CN technique is depicted in FIG. 8 for a single polygon andsingle point located at the end of the arrow labeled “5”. A ray is castfrom the exterior of the polygon to the point being tested and thecounter is incremented for each edge that is crossed. Although in FIG. 8the counter is shown as being incremented in the order in which the rayintersects the polygon's boundary segments, there is no requirement thatthe segments be tested in any particular order. Typically, the linesegments are iterated either clockwise or anti-clockwise from anarbitrary “starting point” in the polygon boundary. The CN techniquedeclares a point to be inside the polygon if the total crossing count isodd. Intuitively this makes sense, because crossing into a boundary fromthe exterior places the ray inside the shape, and the ray can only exitthe shape by crossing the boundary again, thus flipping the count fromodd to even. Note that the ray may be cast in any direction; the choiceof a horizontal ray in FIG. 8 is arbitrary. Furthermore, it does notmatter whether the ray is cast from the point to infinity in anydirection, or from infinity to the point. The crossing count will be thesame regardless of whether the ray is cast inward or outward.

Because this disclosure focuses on accelerating the CN technique, andbecause the CN technique has the same computational complexity as the WNtechnique, we will not describe the WN technique further other than tosay that it iterates the vertices of the polygon and counts the numberof full turns around the point being tested.

There are a number of existing approaches to accelerating PIP testing.Perhaps the most common is to insert the polygons into a spatial datastructure such as an R-Tree. One example of this is the PostGIS™database. The R-Tree can be queried to retrieve a set of candidatepolygons based on rectangular bounding boxes surrounding each polygon.This eliminates polygons whose bounding boxes do not surround the pointin question. Each surviving candidate must then be tested using eitherCN or WN to determine if the point in question falls inside, outside oron the edge of the candidate. Because candidates may contain largenumbers of vertices, and because there may exist many candidates, the CNand WN techniques are still the bottleneck because they are both O(n)where n is the number of vertices in the polygon being tested. Morerecently, an approach has been proposed that builds an indexspecifically for PIP testing; however, this approach suffers from anexponential index build complexity of O(n^(2.6)). Both the R-Tree andthe above-described index-building approach rely on in-memory treestructures. The performance of these in-memory tree structures whenstored on spinning disk media as opposed to RAM, depends on the depth ofthe tree because each link traversal in a tree search requires a seekoperation, and seek operations are slow on traditional spinning media.

The new indexing technique described in this disclosure involves castinghorizontal rays from polygon segments onto a reference line and thenstoring the set of segments intersected by the rays. For a given pointto be PIP tested, the set of intersected segments across all polygons inthe index can then be retrieved from the index, and fed directly to theCN technique. This obviates the need to iterate the boundaries of thepolygons to determine which segments intersect a ray emanating from thepoint to be tested.

FIG. 9 shows a polygon labeled “A”, and the projection of line segmentsS1 through S4 onto the y-axis. Note that segment projections overlap onthe y-axis, and the set of overlapping segments is constant over anycontinuous range of y values bounded by the projection of 2 successivepoints on the y-axis. For instance, referring to FIG. 9, for thecontinuous range of y values [y2, y3) one can see that a horizontal rayfixed at any y value in the range will intersect the set of segments[S2, S4]. Therefore, an indexing mechanism can be designed such that fora given point to be PIP tested, the y coordinate of the point is used tolook up a corresponding set of intersected segments based upon which ofa finite set of y-ranges the point's y value falls inside.

TABLE 1 {S1, --, y1, O} {S4, --, y1, O} {S1, --, y2, C} {S2, --, y2, O}{S3, --, y3, O} {S4, --, y3, C} {S2, --, y4, C} {S3, --, y4, C}

We now describe how to build such an index that comprises sortedy-ranges, and their corresponding set of intersected segments. Considera point data structure for describing the points as {segmentId, x, y,order}. The “order” field can assume the values O or C for “opening” or“closing”. An “opening” point for a segment corresponds to the pointwith the smaller Y-coordinate. A “closing” point is the segment's pointwith the larger Y-coordinate. For example, the coordinate of S1 with thesmaller y value would be {S1, --, y1, O} (the x values are shown as “--”). By sorting the point structures based on their y value fromsmallest to largest, we obtain the ordering with ascending y valuesillustrated in Table 1.

By traversing this sorted set of point data structures from beginning toend, we can create a corresponding list of open segments (that is tosay, segments that would intersect a horizontal ray projected from pointx,y). As we traverse the listed point structures shown in Table 1, wecan perform the following operation: if the order field is “O”, then weadd the corresponding segment ID to the list of open segments. If theorder field is “C”, then we remove the corresponding segment ID from thelist. Hereafter, we call this the “opening and closing technique.”Following this simple procedure, the list of open segments is shown nextto each point data structure as is illustrated in Table 2 below.

TABLE 2 {S1, --, y1, O}: [S1] {S4, --, y1, O}: [S1, S4] {S1, --, y2, C}:[S4] {S2, --, y2, O}: [S2, S4] {S3, --, y3, O}: [S2, S3, S4] {S4, --,y3, C}: [S2, S3] {S2, --, y4, C}: [S3] {S3, --, y4, C}: [ ]

For the list shown in Table 2, the final list for a given y valueprovides the set of open segments between the current y value and thenext y value. Table 3 illustrates this by striking out the non-finalsegments.

From the remaining adjacent y values (those not struck out), we canextract the y coordinates, and the open lists to create continuous yranges, and corresponding lists of open segments, as shown in Table 4,including the ranges from negativity infinity, and to positive infinityon the y-axis.

TABLE 3

{S4, --, y1, O}: [S1, S4]

{S2, --, y2, O}: [S2, S4]

{S4, --, y3, C}: [S2, S3]

{S3, --, y4, C}: [ ]

TABLE 4 [−INF, y1): [ ] [y1, y2): [S1, S4] [y2, y3): [S2, S4] [y3, y4):[S2, S3] [y4, INF): [ ]

FIG. 10 illustrates each of the 5 ranges, and a horizontal ray castthrough each range. One can easily validate from the image in FIG. 10that the list of open segments for each range, as shown in Table 4,accurately describes the set of segments intersected by a given ray. Forexample, the ray in range [y3, y4) intersects S2 and S3. For each rangeof y-values, we have computed the set of intersections of a horizontalray. Therefore, the list of open segments is actually a set of rayintersections for any ray in the given range.

We can easily extend the above-described methodology to multiplepolygons by including the identity of the polygon in the segmentIdfield. For example, FIG. 11 illustrates the case in which multiplepolygons are projected onto the y-axis. Polygon A has segments withidentity S1A, S2A, and S3A, and Polygon B has segments S1B, S2B, andS3B. Applying the above-described technique to the polygons shown inFIG. 10 yields the result that is displayed in Table 5. A comparison ofTable 3 and Table 5 shows that, as one would expect, as more polygonsare present, the set of ray intersections becomes larger for a givenrange.

TABLE 5

{S2A, --, y1, O}: [S1A, S2A]

{S1A, --, y2, C}: [S2A, S3A]

{S3B, --, y3, O}: [S2A, S3A, S1B, S3B]

{S1B, --, y4, C}: [S2A, S3A, S3B, S2B]

{S3B, --, y5, C}: [S2A, S3A]

{S3A, --, y6, C}: [ ]

We now describe a technique for minimizing the storage of rayintersection lists because the lists themselves can grow large and tendto be duplicative with respect to their neighboring lists. This isaccomplished by storing 1 of every K ray intersection lists, and usingthe point data structure to update a stored intersection list atruntime. That is, the intersection list is stored modulo K. In thisvariation, the point data structure must be stored for each point and isused at runtime to update a stored ray intersection set to the desiredy-range. For example, Table 6 below illustrates polygons of FIG. 11stored with K=3 (modulo 3). Note how only one of every threeintersection sets is not struck out, and also note that no pointstructures are struck out. The modulo parameter K provides a practicaltradeoff between space in the index and the runtime cost of updating astored ray to the desired y-range. In practice, a K value as large as1024 can be used. For very large K values the storage space required forthe index is approximately the same as for the polygons themselves.

We now describe the process of determining the set of intersectedsegments of a horizontal ray at an arbitrary height, given the modulostyle of index storage. Given a desired ray height of y, the sortedpoint structure is binary searched to determine the point having astored ray, and a y value less than or equal to the desired y value. Inessence, the system retrieves a ray that is parallel to, but lower thanthe desired ray. This lower ray is then raised to the desired heightusing the opening and closing technique. When a point is reached havinga y value greater than the desired height, the technique terminates.

TABLE 6 {S1A, --, y1, O}: [S1A] {S2A, --, y1, O}: 

{S3A, --, y2, O}: 

{S1A, --, y2, C}: [S2A, S3A,] {S1B, --, y3, O}: 

{S3B, --, y3, O}: 

{S2B, --, y4, O}: [S2A, S3A, S1B, S3B, S2B] {S1B, --, y4, C}: 

{S2B, --, y5, C}: 

{S3B, --, y5, C}: [S2A, S3A] {S2A, --, y6, C}: 

{S3A, --, y6, C}: 

For example, consider a point at location {x,y} for which we want toanswer the PIP question. Assume y falls in the range [y4, y5) of Table6. Binary searching the point structure yields {S2B, --, y4, O}: [S2A,S3A, S1B, S3B, S2B] because the point structure and ray are lower thanor equal to the desired y. Correcting the ray using the following pointstructure {S1B, --, y4, C} updates the ray to [S2A, S3A, S3B, S2B],which is the correct set of intersections for a ray falling in the range[y4, y5). The next point structure, {S2B, --, y5, C} causes the updatingto stop, because y5 is greater than the desired height.

The preceding disclosure describes how to retrieve a pre-cast ray fromthe index. Also, for the case of modulo indexing, we showed how to“correct” a stored ray to raise its height to the desired range. We nowdescribe the step of how to apply the CN technique to the ray. The raysretrieved (and possibly corrected) from the index are cast from negativeinfinity in the X direction toward positive infinity in the X direction.The intersection sets, such as [S2A, S3A, S1B, S3B, S2B] are made ofsegment IDs that include the polygon identity (for instance “A” and“B”). Therefore, before feeding segments to the CN technique, we mustfirst group them by polygonId. This is illustrated in FIG. 12. While wehave heretofore described the elements of the intersection list assimple lists of segmentId values, we in fact store both the opening andclosing point of each segment in conjunction with the segmentId. Thereason for this is that a ray cast from negative infinity to positiveinfinity effectively includes two ray tests (we can think of this as aray originating at the point to test, and emanating to positive infinityand the second ray originating at the point to test emanating tonegative infinity). Because the CN test casts a single ray, we need thetwo points defining each line segment to determine which segments do notintersect the test ray, which we arbitrarily choose to be the oneemanating to positive infinity.

Using the data illustrated in FIG. 12, the CN test first groups thesegments by polygon. Then, for each set, the CN test counts segmentsthat are intersected by the ray emanating to positive infinity. Fromthis point, the output of the CN technique is of the standard form: ifthe count is odd, the point in question was inside the given polygon. Ifthe count is even, the point in question was outside the given polygon.

In the technique presented so far, a ray must be retrieved (andcorrected as needed) for each point to be tested. In practice,retrieving the ray and correcting it takes substantially longer (byorders of magnitude) than performing the CN technique on the retrievedset. The fact that the intersection set is invariant for a given range[ymin, ymax) allows for amortization of the intersection set retrievaland correction costs when a batch of points needs to be PIP tested. Thiscan be accomplished by first sorting the points to be batch tested basedon their y value. While processing the first point in the now-sortedbatch, the corresponding ray can be retrieved and corrected from theindex. For the second point, and all subsequent points to be tested, theray can be incrementally corrected. That is to say, if the subsequentpoint to be tested still falls within the [ymin, ymax) range of thecorrected ray, then no further action is required, and the CN technique,is again applied directly to the subsequent point. Subsequent points areprocessed until a point is encountered whose y value surpasses the upperextent of the ray's valid range (ymax). At this point, rather thanperforming a brand new query for a ray (which would require a binarysearch), one can simply continue to apply correction to the ray whichrequires forward iteration in the index, but not binary searching.

Note that in addition to determining which points fall within eachpolygon, the above-described technique can be modified to determinewhether a given shape overlaps, intersects or encloses another shape byperforming the technique for the points that define the given shape.

Process of Displaying Geographic Data

FIG. 13 presents a flow chart illustrating the process of displayinggeographic data in accordance with the disclosed embodiments. At thestart of the process, the system obtains a set of polygons that define aset of geographic regions, wherein each polygon comprises line segmentsthat define a border of the polygon, and wherein each line segment isdefined by coordinates for two endpoints of the line segment (step1302). Next, the system projects rays from the endpoints of the linesegments that comprise the set of polygons onto a reference line,wherein the rays are projected in a direction orthogonal to thereference line to form intersection points with the reference line (step1304). (In some embodiments, the reference line is the y-axis in aCartesian coordinate system and the rays are projected horizontallyparallel to the x-axis.) Then, for each interval between pairs ofconsecutive intersection points on the reference line, the system keepstrack of open line segments that project onto the interval (step 1306).

Next, for each data point in a set of data points to be processed, thesystem identifies a relevant interval on the reference line that thedata point projects onto (step 1308), and performs a crossing number(CN) operation by counting intersections between a ray projected fromthe data point and open line segments associated with the relevantinterval to identify zero or more polygons that the data point fallsinto (step 1310). The intersections can be detected by performing anintersection test using an equation for the line segment (y=mx+b) to seewhether a ray projected from positive infinity to the data pointintersects the line segment. If the reference line is the y-axis, thesystem can perform a simple initial filtering test to see whether both xcoordinates of the line segment are less than the data point's xcoordinate. This enables the system to throw out line segments that areobviously not intersected by the ray. The system then increments a countfor each polygon that the data point falls into (step 1312).

Finally, the system displays the set of geographic regions, wherein eachpolygon that defines a geographic region is marked to indicate a numberof data points that fall into the polygon (step 1314).

Process of Performing a Geofencing Operation

FIG. 14 presents a flow chart illustrating the process of performing ageofencing operation in accordance with the disclosed embodiments. Atthe start of this process, the system obtains a set of polygons thatdefine a set of geographic regions, wherein each polygon comprises linesegments that define a border of the polygon, and wherein each linesegment is defined by coordinates for two endpoints of the line segment(step 1402). Next, the system projects rays from the endpoints of theline segments that comprise the set of polygons onto a reference line,wherein the rays are projected in a direction orthogonal to thereference line to form intersection points with the reference line (step1404). Then, for each interval between pairs of consecutive intersectionpoints on the reference line, the system keeps track of open linesegments that project onto the interval (step 1406).

Next, for a data point to be processed, the system identifies a relevantinterval on the reference line that the data point projects onto (step1408). The system subsequently performs a crossing number (CN) operationby counting intersections between a ray projected from the data pointand open line segments associated with the relevant interval to identifyzero or more polygons that the data point falls into (step 1410).Finally, the system performs a geofencing operation based on theidentified zero or more polygons that the data point falls into (step1412). For example, the geofencing operation can involve sending anotification to a user when the user's location (obtained from theuser's phone) indicates that the user has crossed a geofence boundaryassociated with a restricted security area.

Processing a Query Involving Geographic Information

FIG. 15 presents a flow chart illustrating the processing of a queryinvolving geographic information in accordance with the disclosedembodiments. First, the system receives the query to be processed,wherein the query is associated with a set of geographic regions (step1502). Next, the system uses a late-binding schema generated from thequery to retrieve data points from a set of events containing previouslygathered data (step 1504). For example, if the query asks to count thepopulation in each state in the United States, the late-binding schemacan retrieve residence locations for people from a set of event data.Then, for each for each data point in a set of data points, the systemidentifies zero or more geographic regions in the set of geographicregions that the data point falls into (step 1506). Finally, the systemdisplays the set of geographic regions, wherein each polygon thatdefines a geographic region is marked to indicate a number of datapoints that fall into the polygon (step 1508).

The preceding description was presented to enable any person skilled inthe art to make and use the disclosed embodiments, and is provided inthe context of a particular application and its requirements. Variousmodifications to the disclosed embodiments will be readily apparent tothose skilled in the art, and the general principles defined herein maybe applied to other embodiments and applications without departing fromthe spirit and scope of the disclosed embodiments. Thus, the disclosedembodiments are not limited to the embodiments shown, but are to beaccorded the widest scope consistent with the principles and featuresdisclosed herein. Accordingly, many modifications and variations will beapparent to practitioners skilled in the art. Additionally, the abovedisclosure is not intended to limit the present description. The scopeof the present description is defined by the appended claims.

The data structures and code described in this detailed description aretypically stored on a computer-readable storage medium, which may be anydevice or medium that can store code and/or data for use by a system.The computer-readable storage medium includes, but is not limited to,volatile memory, non-volatile memory, magnetic and optical storagedevices such as disk drives, magnetic tape, CDs (compact discs), DVDs(digital versatile discs or digital video discs), or other media capableof storing code and/or data now known or later developed.

The methods and processes described in the detailed description sectioncan be embodied as code and/or data, which can be stored on anon-transitory computer-readable storage medium as described above. Whena system reads and executes the code and/or data stored on thenon-transitory computer-readable storage medium, the system performs themethods and processes embodied as data structures and code and storedwithin the non-transitory computer-readable storage medium.

Furthermore, the methods and processes described above can be includedin hardware modules. For example, the hardware modules can include, butare not limited to, application-specific integrated circuit (ASIC)chips, field-programmable gate arrays (FPGAs), and otherprogrammable-logic devices now known or later developed. When thehardware modules are activated, the hardware modules perform the methodsand processes included within the hardware modules.

What is claimed is:
 1. A computer-implemented method for displayinggeographic data, comprising: obtaining a set of polygons that define aset of geographic regions, wherein each polygon comprises line segmentsthat define a border of the polygon, and wherein each line segment isdefined by coordinates for two endpoints of the line segment; projectingrays from the endpoints of the line segments that comprise the set ofpolygons onto a reference line, wherein the rays are projected in adirection orthogonal to the reference line to form intersection pointswith the reference line; for each interval between pairs of consecutiveintersection points on the reference line, keeping track of open linesegments that project onto the interval; for each data point in a set ofdata points to be processed, identifying a relevant interval on thereference line that the data point projects onto, performing a crossingnumber (CN) operation by counting intersections between a ray projectedfrom the data point and open line segments associated with the relevantinterval to identify zero or more polygons that the data point fallsinto, and incrementing a count for each polygon that the data pointfalls into; and displaying the set of geographic regions, wherein eachpolygon that defines a geographic region is marked to indicate a numberof data points that fall into the polygon.
 2. The computer-implementedmethod of claim 1, wherein displaying the set of geographic regionsincludes displaying a choropleth map wherein each geographic region isshaded, patterned or colored in proportion to a number of data pointsthat fall into the geographic region.
 3. The computer-implemented methodof claim 1, wherein the reference line is an X coordinate axis.
 4. Thecomputer-implemented method of claim 1, wherein the reference line is aY coordinate axis.
 5. The computer-implemented method of claim 1,wherein determining the relevant interval for the data point includesperforming a binary search through an array containing the intersectionpoints on the reference line.
 6. The computer-implemented method ofclaim 1, wherein keeping track of the open line segments that projectonto the intervals includes maintaining a list of open line segments forevery nth interval on the reference line.
 7. The computer-implementedmethod of claim 1, wherein keeping track of the open line segments thatproject onto the intervals includes maintaining a list of open linesegments for every nth interval on the reference line; and whereinidentifying a relevant interval for a data point includes obtaining alist of open intervals for a nearest nth interval on the reference line,and adjusting the list to account for intervening line segment openingsand closings associated with intersection points located between thenearest nth interval and the relevant interval on the reference line toproduce a list of open intervals for the relevant interval.
 8. Thecomputer-implemented method of claim 1, wherein prior to performing theCN operation for each data point, the set of data points is sorted basedon an ordering of projections of the set of data points on the referenceline; and wherein the set of data points are processed in the sortedorder to facilitate reducing an average amount of time required toidentify a relevant interval for each data point.
 9. Thecomputer-implemented method of claim 1, wherein prior to processing theset of data points, the method further comprises obtaining the set ofdata points by: receiving a query to be processed, wherein the query isassociated with the set of geographic regions; and using a late-bindingschema generated from the query to retrieve the set of data points froma set of events containing previously gathered data.
 10. Thecomputer-implemented method of claim 1, wherein obtaining the set ofpolygons includes retrieving the set of polygons from a geographicdatabase.
 11. A non-transitory computer-readable storage medium storinginstructions that when executed by a computer cause the computer toperform a method for displaying geographic data, the method comprising:obtaining a set of polygons that define a set of geographic regions,wherein each polygon comprises line segments that define a border of thepolygon, and wherein each line segment is defined by coordinates for twoendpoints of the line segment; projecting rays from the endpoints of theline segments that comprise the set of polygons onto a reference line,wherein the rays are projected in a direction orthogonal to thereference line to form intersection points with the reference line; foreach interval between pairs of consecutive intersection points on thereference line, keeping track of open line segments that project ontothe interval; for each data point in a set of data points to beprocessed, identifying a relevant interval on the reference line thatthe data point projects onto, performing a crossing number (CN)operation by counting intersections between a ray projected from thedata point and open line segments associated with the relevant intervalto identify zero or more polygons that the data point falls into, andincrementing a count for each polygon that the data point falls into;and displaying the set of geographic regions, wherein each polygon thatdefines a geographic region is marked to indicate a number of datapoints that fall into the polygon.
 12. The non-transitorycomputer-readable storage medium of claim 11, wherein displaying the setof geographic regions includes displaying a choropleth map wherein eachgeographic region is shaded, patterned or colored in proportion to anumber of data points that fall into the geographic region.
 13. Thenon-transitory computer-readable storage medium of claim 11, wherein thereference line is an X coordinate axis.
 14. The non-transitorycomputer-readable storage medium of claim 11, wherein the reference lineis a Y coordinate axis.
 15. The non-transitory computer-readable storagemedium of claim 11, wherein determining the relevant interval for thedata point includes performing a binary search through an arraycontaining the intersection points on the reference line.
 16. Thenon-transitory computer-readable storage medium of claim 11, whereinkeeping track of the open line segments that project onto the intervalsincludes maintaining a list of open line segments for every nth intervalon the reference line.
 17. The non-transitory computer-readable storagemedium of claim 11, wherein keeping track of the open line segments thatproject onto the intervals includes maintaining a list of open linesegments for every nth interval on the reference line; and whereinidentifying a relevant interval for a data point includes obtaining alist of open intervals for a nearest nth interval on the reference line,and adjusting the list to account for intervening line segment openingsand closings associated with intersection points located between thenearest nth interval and the relevant interval on the reference line toproduce a list of open intervals for the relevant interval.
 18. Thenon-transitory computer-readable storage medium of claim 11, whereinprior to performing the CN operation for each data point, the set ofdata points is sorted based on an ordering of projections of the set ofdata points on the reference line; and wherein the set of data pointsare processed in the sorted order to facilitate reducing an averageamount of time required to identify a relevant interval for each datapoint.
 19. The non-transitory computer-readable storage medium of claimof claim 11, wherein prior to processing the set of data points, themethod further comprises obtaining the set of data points by: receivinga query to be processed, wherein the query is associated with the set ofgeographic regions; and using a late-binding schema generated from thequery to retrieve the set of data points from a set of events containingpreviously gathered data.
 20. The non-transitory computer-readablestorage medium of claim of claim 11, wherein obtaining the set ofpolygons includes retrieving the set of polygons from a geographicdatabase.
 21. A system that displays geographic data, comprising: atleast one processor and at least one associated memory; and a displaymechanism that executes on the at least one processor and is configuredto: obtain a set of polygons that define a set of geographic regions,wherein each polygon comprises line segments that define a border of thepolygon, and wherein each line segment is defined by coordinates for twoendpoints of the line segment; project rays from the endpoints of theline segments that comprise the set of polygons onto a reference line,wherein the rays are projected in a direction orthogonal to thereference line to form intersection points with the reference line; foreach interval between pairs of consecutive intersection points on thereference line, keep track of open line segments that project onto theinterval; for each data point in a set of data points to be processed,identify a relevant interval on the reference line that the data pointprojects onto, perform a crossing number (CN) operation by countingintersections between a ray projected from the data point and open linesegments associated with the relevant interval to identify zero or morepolygons that the data point falls into, and increment a count for eachpolygon that the data point falls into; and display the set ofgeographic regions, wherein each polygon that defines a geographicregion is marked to indicate a number of data points that fall into thepolygon.
 22. The system of claim 21, wherein while displaying the set ofgeographic regions, the display mechanism is configured to display achoropleth map wherein each geographic region is shaded, patterned orcolored in proportion to a number of data points that fall into thegeographic region.
 23. The system of claim 21, wherein the referenceline is an X coordinate axis.
 24. The system of claim 21, wherein thereference line is a Y coordinate axis.
 25. The system of claim 21,wherein while determining the relevant interval for the data point, thedisplay mechanism is configured to perform a binary search through anarray containing the intersection points on the reference line.
 26. Thesystem of claim 21, wherein while keeping track of the open linesegments that project onto the intervals, the display mechanism isconfigured to maintain a list of open line segments for every nthinterval on the reference line.
 27. The system of claim 21, whereinwhile keeping track of the open line segments that project onto theintervals, the display mechanism is configured to maintain a list ofopen line segments for every nth interval on the reference line; andwherein while identifying a relevant interval for a data point, thedisplay mechanism is configured to, obtain a list of open intervals fora nearest nth interval on the reference line, and adjust the list toaccount for intervening line segment openings and closings associatedwith intersection points located between the nearest nth interval andthe relevant interval on the reference line to produce a list of openintervals for the relevant interval.
 28. The system of claim 21, whereinprior to performing the CN operation for each data point, the displaymechanism is configured to sort the set of data points based on anordering of projections of the set of data points on the reference line;and wherein the display mechanism is configured to process the set ofdata points in the sorted order to facilitate reducing an average amountof time required to identify a relevant interval for each data point.29. The system of claim 21, wherein prior to processing the set of datapoints, the system is further configured to: receive a query to beprocessed, wherein the query is associated with the set of geographicregions; and use a late-binding schema generated from the query toretrieve the set of data points from a set of events containingpreviously gathered data.
 30. The system of claim 21, wherein whileobtaining the set of polygons, the system is configured to retrieve theset of polygons from a geographic database.